Neural Network Compression Framework for Fast Model Inference

We present a new PyTorch-based framework for neural network compression with fine-tuning named Neural Network Compression Framework (NNCF) (https://github.com/openvinotoolkit/nncf) . It leverages recent advances of various methods and implements some them, namely quantization, sparsity, filter pruning binarization. These allow producing more hardware-friendly models that can be efficiently run on general-purpose hardware computation units (CPU, GPU) or specialized deep learning accelerators. show the implemented their combinations successfully applied to wide range architectures tasks accelerate inference while preserving original model’s accuracy. The used in conjunction supplied training samples as standalone package seamlessly integrated into existing code minimal adaptations.